Counterexample-Guided Abstraction Refinement for POMDPs

We study a sound and complete counterexampleguided abstraction refinement (CEGAR) framework for partially observable Markov decision processes (POMDPs). This framework allows automatic reasoning to find a proper abstraction for POMDPs and reduce model checking complexity. A safety fragment of Probabilistic Computation Tree Logic (PCTL), safePCTL, with the finite horizon is considered as system specification. As the abstraction for POMDPs, z-labeled 0/1-weighted automata (0/1-WA) are extended from the weighted automata by defining the observation labeling function for discrete states. We then propose a simulation relation, safe simulation relation, for 0/1-WA and prove the preservation of safe-PCTL by the safe simulation. With 0/1-WA and safe simulation relation, we further address a novel CEGAR framework to find a proper 0/1WA as the abstraction of POMDP. Initially, we start from the coarsest abstract system generated from a quotient construction and iteratively check the satisfaction relation of the given specification on the abstract system. Counterexamples from model checking on the abstract system are derived in the forms of a set of paths that violate the specification with enough accumulative probability. Given these counterexamples, we verify whether or not these counterexamples are real witnesses for violation of specification on the concrete system. If not, we use these spurious counterexamples and refine the quotient construction to update the abstract system until satisfaction relation is proved to be true or real counterexample has been found for the original concrete POMDP.